User selectable focus regions in an image capturing device

ABSTRACT

An image capturing device includes a focusable lens apparatus, an image sensor comprising a plurality of pixel elements, and an electronically actuatable shutter device including a plurality of individually addressable and actuatable shutter elements. A shutter element substantially corresponds to one or more pixel elements. A processor controls a focus depth of the lens apparatus and selectively actuates particular shutter elements associated with each of a plurality of focus depths during image capture.

FIELD OF THE INVENTION

The present invention relates generally to an image capturing device,and more particularly to an image capturing device capable of capturingan image at multiple focus depths.

BACKGROUND OF THE INVENTION

Still image capturing devices are used to visually memorialize scenes,events, or items. Still image capturing devices, such as analog anddigital cameras, include a lens, a shutter, and some manner of imagesensor. In addition, most modern cameras include a processor and/orother control electronics that function to control shutter speed,aperture, flash, focus, etc.

The shutter and the image sensor are the main components of a stillimage capturing device and operate together in order to produce animage. In operation, the shutter is opened briefly to expose the imagesensor to focused light from a lens and thereby form an image. Theoperation of the shutter is very important, and the quality of thecaptured image depends on a proper exposure time based on lighting,movement of the subject, focus distance, etc.

A prior art shutter approach used a mechanical shutter. The mechanicalshutter has been widely used for a number of years and is generally inthe form of an iris-type shutter. However, the prior art mechanicalshutter has many drawbacks, including weight, large size, susceptibilityto dirt and wear, and the difficulty of precisely controlling shutterexposure times over a wide range of conditions. In addition, themechanical shutter exposes the entire image as a unit and essentially atonce (however, the iris mechanism is open at the center for a longerlength of time than at the peripheral region of the iris).

In some prior art cameras, the mechanical shutter is electronicallyactivated by a motor or other electrical actuator. This may produce amore accurate shutter control, but consumes a lot of electrical power,is inflexible, and still exposes the entire image as a unit and for anessentially constant duration.

The prior art therefore cannot control a focus depth of an image inorder to capture image portions or objects at different focus depths.For example, in an image of a person in front of a background, prior artimage capturing devices focus to the person, and the entire image iscaptured at that single focus depth. As a result, the background is outof focus. An image captured according to the prior art therefore mayhave portions that are in focus and may have portions that are out offocus.

Therefore, there remains a need in the art for improvements in stillimage capturing devices.

SUMMARY OF THE INVENTION

A still image capturing device includes a focusable lens apparatuscapable of being focused over a range of focus depths, an image sensorcomprising a plurality of pixel elements, and an electronicallyactuatable shutter device comprising a plurality of individuallyaddressable and actuatable shutter elements. A shutter elementsubstantially corresponds to one or more pixel elements. A processorcontrols a focus depth of the lens apparatus and selectively actuatesparticular shutter elements associated with each of a plurality of focusdepths during image capture.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a still image capturing device according toone embodiment of the invention;

FIG. 2 shows detail of the shutter array and an electronic image sensor;

FIG. 3 is a flowchart of an image capturing method according to oneembodiment of the invention; and

FIG. 4 is a flowchart of an image capturing method according to anotherembodiment of the invention.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of a still image capturing device 100according to one embodiment of the invention. The image capturing device100 includes a lens apparatus 102, a processor 106, a shutter array 114,an image sensor 117, a shutter button 118, and a memory 120. The shutterarray 114 and the image sensor 117 may be provided as an imaging module110. In addition, the image capturing device 100 includes a userinterface 138.

The shutter array 114 is electronically actuated and comprises a twodimensional array of individually addressable shutter elements (see FIG.2 and accompanying discussion). The shutter array 114 is preferably aliquid crystal display (LCD) element comprising a two-dimensional arrayof individually addressable and actuatable shutter elements.Alternatively, the shutter array 114 may be a switchable reflectingdevice, such as a microelectromechanical device comprising atwo-dimensional array of electrically addressable and actuatablemechanical shutter sub-elements.

The image sensor 117 may comprise an electronic image sensor orconventional film. The film may comprise a plurality of pixel elements(i.e., picture elements) to be exposed. Preferably, the image sensor 117comprises a CCD array or a CMOS array, for example, and is formed of atwo-dimensional array of pixel elements. The pixel elements receivelight through the shutter array 114 and generate electronic signalsrelated to the amount of light received.

The processor 106 may be any type of general purpose processor and maycontrol the overall operation of the image capturing device 100. Theprocessor 106 receives input signals from the shutter button 118 and theuser interface 138 and controls a shuttering operation of the shutterarray 114 in order to capture an image. In addition, if the image sensor117 is an electronic device, the processor 106 controls the storage ofdigital image data produced by the image sensor 117. For example, theprocessor 106 may receive images and store them in the memory 120.Alternatively, the processor 106 may control camera operation in orderto capture an image on film. In addition, the processor 106 receivesuser input signals and performs functions specified by the user inputsignals.

The focus rangefinder 123 may be any type of device capable of measuringa distance from the image capturing device 100 to a subject. The focusrangefinder 123 may be an ultrasound or infrared distance measuringdevice, for example, or alternatively may employ radio waves. The focusrangefinder 123 may provide one or more range measurements to theprocessor 106, and optionally may be able to obtain a plurality of rangemeasurements over an area to be captured during an image captureprocess.

Alternatively, the range measurements may be generated by the processor106 if the image sensor 117 is an electronic device such as a CCD orCMOS array, for example. This is the most commonly used method in bothdigital and film image capturing devices. In this embodiment, allshutter elements 204 of the shutter array 114 (see FIG. 2) may bemomentarily opened such that the processor 106 receives an imagecomprising a two-dimensional array of pixel values (i.e., apre-exposure). The processor 106 may use an image analysis algorithm tofind edges in the resultant image and determine whether the edges arefocused by determining whether the edges are sharply defined.Furthermore, by changing the focus depth of the lens apparatus 102 andperforming repeated image capture and image analysis steps, theprocessor 106 may determine all focus depths of objects or regions inthe image.

The user interface 138 may include a display device and a user inputdevice. The display device may be any type of electronic display device,such as an LCD screen, for example. The user input device may compriseone or more buttons. The one or more buttons may be used in conjunctionwith a displayed menu or other selection display, and a user maymanipulate the buttons in order to make selections. Alternatively, theuser interface 138 may comprise a touch screen wherein the user maytouch displayed icons, symbols, etc., in order to make selections and tocontrol operation of the image capturing device 100.

In operation, the shutter array 114 is controlled by the processor 106in response to a press of the shutter button 118. Incoming light entersthe image capturing device 100 through the lens apparatus 102 andimpinges upon the shutter array 114. When activated by the processor106, the shutter array 114 allows the incoming light to pass through,i.e., the shutter array 114 transforms from a light reflective or opaquestate to a light transmissive state. The shutter array 114 is controlledto be light transmissive for a predetermined exposure period, and iscontrolled to become light opaque or reflective at the end of thepredetermined exposure period. Therefore, when the light passes throughthe shutter array 114 and impinges on the image sensor 117, an image maybe captured by the image sensor 117.

The memory 120 may be any type of memory, including all types of randomaccess memory (RAM), read-only memory (ROM), flash memory, magneticstorage media such as magnetic disc, tape, etc., or optical or bubblememory. In a digital image capturing device, the memory 120 may include,among other things, an image storage cell 131, an optional objectsstorage cell 141, an image map storage cell 162, and an optional rangestorage cell 167. In addition, the memory 120 may store software orfirmware to be executed by the processor 106.

The image storage cell 131 may store one or more complete images. Inaddition, it may store image portions obtained during an image capture,such as the image portions 1 and 2 shown. The image portions may besuccessively captured at different focus depths in order to capture animage that is substantially in focus over its entire area. The imageportions may be captured by successively focusing the lens apparatus 102at various focus depths and exposing distinct regions of the imagesensor 117, as defined by the image map 162.

The image map storage cell 162 may store one or more image maps thatrelate an object or focus depth to a particular grouping of shutterelements 204. Therefore, each image map stored in the image map storagecell 162 controls actuation of a predetermined grouping of shutterelements 204. By employing multiple image maps, an image capture may beconducted by successively opening each grouping of shutter elements, inconjunction with moving the lens apparatus 102 to achieve eachcorresponding focus depth.

The objects storage cell 141 is optional and may store definitions ofone or more objects to be included in an image capture. The objectdefinitions are generally temporary in nature and may only need to bestored until the associated image capture has been completed. Theobjects may be defined by focus depth measurements, by boundariesentered by the user, or may be one of several predetermined, storedshapes selected by the user.

The range storage cell 167 may store one or more focus depth rangemeasurements. The range measurements are generally temporary in natureand may only need to be stored until the associated image capture hasbeen completed. The range measurements are generated by the focusrangefinder 123 and may be used to capture individual image portions inorder to create an image composed of a plurality of image portions.

FIG. 2 shows detail of the shutter array 114 and an electronic imagesensor 117. A two-dimensional array of shutter elements 204 is formed onor assembled to a two-dimensional array of image sensor pixel elements207. Therefore, in one embodiment a shutter element 204 may correspondsubstantially in size to a pixel element 207. Alternatively, in anotherembodiment the shutter element 204 may correspond in size to two or morepixel elements 207, allowing a shutter element 204 to shutter two ormore pixel elements 207. Furthermore, the shutter array 114 is arrangedso that the shutter elements 204 are substantially aligned with one ormore corresponding pixel elements 207, and may operate to block ortransmit light to the one or more corresponding pixel elements 207.

As previously discussed, the shutter array 114 may comprise an LCDelement formed of a two-dimensional array of individually addressableand actuatable shutter elements. Therefore, the processor 106 mayactuate one or more shutter elements 204, may actuate a pattern ofshutter elements 204, or may actuate shutter elements 204 for differingperiods of time. The shutter elements 204 may be actuated in anycombination, and may be actuated according to a pattern or timetable.Therefore, the imaging module 110 according to the invention is capableof performing a pixel-by-pixel shuttering. The invention may exposesmall regions, even individual pixel elements, because the shutter array114 may be formed on the electronic image sensor 117. As a result, theshutter array 114 may control exposure of the pixel elements 207 withoutany significant shuttering overlap, light leakage, loss of focus, etc.

In one embodiment, the LCD element is a polymer dispersed liquid crystal(PDLC) element. The PDLC element may be used without polarizing films.The PDLC element may be used regardless of the polarization effect, oralternatively a PDLC shutter array 114 may be formed of shutter elementspossessing different polarization orientations in order to passsubstantially non-polarized light, as discussed below.

In another embodiment, the LCD element is a nematic or super-twistednematic LCD. In these types of LCD, both sides of the LCD elementinclude a polarizing film. Therefore, the image capture employspolarized light. The polarizing single LCD element configuration is thesimplest and cheapest, and provides the best “dark” state and maytherefore still be preferable even though light is lost by having topass through polarizing films.

The polarizing effect may be negated, however. In this embodiment, apixel unit 222 may comprise a pair of shutter elements 204 and a pair ofpixel elements 207. The paired pixel element configuration of the pixelunit 222 is desirable because of the polarization. Therefore, a pixelunit 222 according to the invention may include a shutter element of afirst polarization orientation and a shutter element of a secondpolarization orientation. The second polarization orientation issubstantially orthogonal to the first polarization orientation. As aresult, the two pixel element polarizations are combined to capturesubstantially non-polarized light, and therefore the imaging module 110as a unit may capture a substantially non-polarized image.

The polarizing film in this embodiment may be formed of narrow bands ofpolarizing material, with each pixel element of a pixel unit 222 beingfrom a separate polarization band (the pixel elements may be separatedby a small distance). The bands may be formed having substantiallyperpendicular polarization orientations. Alternatively, each shutterelement 204 may have a corresponding polarizing element that isdeposited on or otherwise formed on the LCD element.

The above non-polarizing shutter may alternatively be implemented usingtwo LCD elements, a beam splitter, and a beam combiner. The beamsplitter splits the incoming light into two light beams and each beam isseparately directed into one of the two LCD shutters. In thisembodiment, the two LCD shutters polarize the light, and the two LCDshutters are positioned in substantially perpendicular polarizationorientations. The polarized light from each shutter is then directedinto the beam combiner, wherein the two substantially perpendicularpolarized light beams are combined to form a substantially non-polarizedresultant light beam.

In an alternate embodiment, the shutter array 114 may comprise atwo-dimensional array of individually addressable and actuatablereflective micromechanical shutter (MEMS) elements, as is known in theart. The microelectromechanical elements may be actuated by an electriccurrent. Unlike the prior art, however, the microelectromechanicalelements are used as a reflective shutter. The MEMS device comprisesactuatable mirrors that can reflect light straight out from the array orscatter it. As before, the actuation of the micromechanical elements iscontrolled by the processor 106 and may be controlled and actuated toselectively expose regions of the electronic image sensor 117.

FIG. 3 is a flowchart 300 of an image capturing method according to oneembodiment of the invention. In step 301, an image capture is initiated.This is generally done at a press of the shutter button 118.

In step 303, the image capturing device 100 performs a first focus depthmeasurement. This may be done using the focus rangefinder 123 or usingthe image analysis, for example. The first focus depth measurement maybe a distance to a subject of the image capture, such as a measurementof a distance to an object centered in a viewfinder of the imagecapturing device 100. The image capturing device 100 therefore generatesa two-dimensional array of focus depth measurements (i.e., an imagemap).

In step 304, the lens apparatus 102 is moved to focus at the first focusdepth.

In step 306, a first image portion is captured at the first focus depth,corresponding to a first subset of the plurality of picture elements.The first image portion may be, for example, an object or item withincross hairs or designated region of the camera's field of view, forexample. The first image portion is captured by opening and closing onlythose shutter elements 204 corresponding to the designated region at afirst focus depth. The image capture therefore comprises exposing aportion of film or a portion of the electronic image sensor 117. Forexample, the first image portion may expose all pixel outputscorresponding to a focus depth of four to six feet.

In an optional step 309, the image capturing device 100 performs asecond focus depth measurement. This may be done by the user moving across-hairs or pointer to another region and initiating a second focusmeasurement. A user may therefore designate multiple focus regions.Alternatively, the second focus depth may be a default, such as aninfinite or maximum focus depth (i.e., the first focus depth is thedistance to a subject and the second focus depth captures a background).

In step 311, the lens apparatus 102 is moved to focus at the secondfocus depth.

In step 315, a second image portion is captured at the second focusdepth, corresponding to a second subset of the plurality of pictureelements. The second image portion is captured by opening and closingonly those shutter elements 204 corresponding to the designated regionat a second focus depth.

It should be understood that more than two image portions may becaptured. Only two are shown and discussed in order to illustrate theconcept of the invention, i.e., that by using a shutter array 114comprising a plurality of shutter elements 204, the image capturingdevice 100 may capture portions of an image at different focal depths.Therefore, the image capturing device 100 according to the invention mayperform iterative steps of measuring focal depths, moving the lensapparatus 102, and capturing successive image portions in order tocapture a complete image that is substantially focused over an entireimage area.

The above described method also may be employed in a camera usingconventional film. Alternatively, the method may be employed in an imagecapturing device 100 that employs a CMOS image sensor 117. The CMOSimage sensor 117 is capable of capturing partial images. Alternatively,a CCD image sensor 117 may be used, but cannot capture and output apartial image. However, the image capturing device 100 does not have toaccept or store all image output from the image sensor 117. Therefore,in a CCD image sensor embodiment, either partial images may be saved tomemory or multiple images may be captured and combined in memory inorder to form a resultant image that was captured at multiple focusdepths.

FIG. 4 is a flowchart 400 of an image capturing method according toanother embodiment of the invention. In step 402, object designationsare accepted by the image capturing device 100. The object designationsmay be input by a user. If input by the user, the designations may begenerated by employing object selection indicia. The object selectionindicia may be a cross-hairs or pointer that may be moved in a display,such as a display of the user interface 138. Therefore, the user may beable to designate an object by pressing a select button or mode buttonwhen the cross-hairs or pointer is positioned on the object in thedisplay. Alternatively, the user may select objects or regions ofinterest. For example, the user may be able to create a box or shapesurrounding an object. Alternatively, an object or objects may beautomatically generated by the processor 106 using image analysis (i.e.,by detecting edges, for example).

Alternatively, if the image capturing device 100 automatically generatesobject designations, this may be done by measuring focus depths andgrouping areas of substantially similar focus depths as being objects.

In step 406, the image capture is initiated, as previously discussed.

In step 411, a first object is captured by exposing a first subset ofthe pixel elements. The first object may be at a first focus depth.

In step 414, a second object image may be captured by exposing a secondsubset of the pixel elements. The second object may optionally be at asecond focus depth, although it may be at approximately at the samefocus depth as other objects in the image capture. The second objectimage is distinct from the first object and may appear in a differentpart of the image.

It should be understood that the method 400 may be iteratively performedand may be done for more than two objects. Therefore, according to themethod, the image capturing device 100 or the user may designatemultiple objects, the lens apparatus 102 may be moved accordingly, andcorresponding groupings of shutter elements 204 may be opened and closedto capture an image composed of multiple objects and therefore ofmultiple image portions.

The image capturing device and methods discussed above have severaladvantages. The image capturing process may be more finely controlledand the image capture may be conducted at multiple focus depths in orderto achieve an image that is substantially focused throughout. Moreover,the user has more control of the focus of the image capturing device,and can capture images wherein objects are at different focus depths.

1. A still image capturing device, comprising: a focusable lensapparatus capable of being focused over a range of focus depths; animage sensor comprising a plurality of pixel elements; an electronicallyactuatable shutter device comprising a plurality of individuallyaddressable and actuatable shutter elements, with a shutter element ofsaid plurality of individually addressable shutter elementssubstantially corresponding to one or more pixel elements of saidplurality of pixel elements; and a processor communicating with saidlens apparatus, said image sensor, and said shutter device, saidprocessor controlling a focus depth of said lens apparatus andselectively actuating particular shutter elements of said shutter deviceassociated with each of a plurality of focus depths during imagecapture.
 2. The apparatus of claim 1, further comprising: a memoryincluding one or more image storage cells capable of storing one or moreimages or one or more image portions, one or more objects storage cellscapable of storing definitions of one or more objects to be included inan image capture, one or more image map storage cells capable of storingone or more image maps that relate an object or focus depth to aparticular grouping of shutter elements, and one or more range storagecells capable of storing one or more focus depth range measurements;wherein said processor is capable of controlling said plurality ofshutter elements according to an image map or one or more objects storedin said memory.
 3. The apparatus of claim 1, further comprising a focusrangefinder capable of measuring one or more focus depths in an image tobe captured.
 4. The apparatus of claim 1, further comprising at leastone user input device, wherein said at least one user input device iscapable of being manipulated by a user in order to set focus regions inan image to be captured.
 5. The apparatus of claim 1, wherein said imagesensor comprises a two-dimensional array of pixel elements and saidshutter device comprises a two-dimensional array of shutter elements. 6.The apparatus of claim 1, wherein said image sensor is a photographicfilm.
 7. An image capturing method for a still image capturing device,wherein the still image capturing device comprises a shutter devicecomprising a set of individually addressable and actuatable shutterelements, comprising the steps of: initiating an image capture in animage sensor of said image capturing device, with said image sensorcomprising a plurality of pixel elements; obtaining a first focus depth;moving a lens apparatus of said image capturing device to said firstfocus depth; capturing a first image portion corresponding to a firstsubset of said plurality of pixel elements at said first focus depth,wherein said first subset of pixel elements is associated with a firstsubset of said shutter elements and said capturing step comprisesactuating only said first subset of said shutter elements; obtaining asecond focus depth; moving said lens apparatus to said second focusdepth; and capturing a second image portion corresponding to a secondsubset of said plurality of pixel elements at said second focus depth,wherein said second subset of pixel elements is associated with a secondsubset of said shutter elements and said capturing step comprisesactuating only said second subset of said shutter elements, wherein saidfirst subset of shutter elements is different than said second subset ofshutter elements.
 8. The method of claim 7, wherein the obtaining stepsfurther comprise receiving said first and second focus depths from afocus rangefinder sensor.
 9. The method of claim 7, wherein theobtaining steps generate a two-dimensional array of focus depthmeasurements.
 10. The method of claim 7, wherein the method furthercomprises: opening all shutter elements of said shutter device toperform a pre-exposure of said sensor element to capture a pre-image;and performing an image analysis on said pre-image to mathematicallydetermine a focus depth of one or more regions in said pre-image. 11.The method of claim 7, wherein the capturing steps capture a partialimage.
 12. The method of claim 7, wherein the capturing steps capturecomplete images, and wherein portions of said complete images arecombined to form a resultant image.
 13. An image capturing method for astill image capturing device having an image sensor comprising aplurality of pixel elements, comprising the steps of: accepting a firstobject designation corresponding to a first object of an image to becaptured; accepting a second object designation corresponding to asecond object of the image to be captured; in response to a single pressof a shutter button; (a) capturing a first image by exposing a firstsubset of said pixel elements, wherein the first image includes an imageof the first object; and (b) after capturing the first image, capturinga second image by exposing a second subset of said pixel elements,wherein the second image includes an image of the second object andwherein said first subset of pixel elements is different than saidsecond subset of pixel elements.
 14. The method of claim 13, wherein thefirst accepting step comprises designating a grouping of shutterelements substantially corresponding to the first object.
 15. The methodof claim 13, wherein said object designations are entered by a user. 16.The method of claim 15, further comprising the steps of: enabling saiduser to position an object indicia on the first object; and enablingsaid user to select said object by manipulating a select input device.17. The method of claim 13, wherein said object designations areautomatically generated by a processor of said image capturing device,said processor employing an image analysis procedure.
 18. The method ofclaim 13, wherein said first object is at a first focus depth and saidsecond object is at a second focus depth.
 19. The method of claim 13,wherein said first object is at a first focus depth and said secondobject is approximately at said first focus depth.
 20. The method ofclaim 13, wherein a lens apparatus of said image capturing device ismoved before capture of said second image and after capture of saidfirst image.
 21. An image capturing method for a still image capturingdevice, comprising the steps of: accepting object designations of one ormore objects in an image to be captured; initiating an image capture inan image sensor of said image capturing device, with said image sensorcomprising a plurality of pixel elements; capturing a first object imageby exposing a first subset of said pixel elements; and capturing asecond object image by exposing a second subset of said pixel elements,wherein the accepting step comprises designating a grouping of shutterelements substantially corresponding to an object.
 22. The method ofclaim 21, wherein said object designations are entered by a user. 23.The method of claim 22, further comprising the steps of: enabling saiduser to position an object indicia on the first object; and enablingsaid user to select said object by manipulating a select input device.24. The method of claim 21, wherein said object designations areautomatically generated by a processor of said image capturing device,said processor employing an image analysis procedure.
 25. The method ofclaim 21, wherein said first object is at a first focus depth and saidsecond object is at a second focus depth.
 26. The method of claim 21,wherein the initiating step is performed at a press of a shutter buttonof said image capturing device.
 27. The method of claim 21, wherein alens apparatus of said image capturing device is moved before capture ofsaid second image and after capture of said first image.